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Conventions 


Before working through this correspondence text make sure you have 
tead A Guide to the Linear Mathematics Course. Of the typographical con- 
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3.0 INTRODUCTION 


This unit is more down-to-earth than the last two. It is concerned with two 
main topics. First (in Section 3.1), it looks at the practical details of how 
vectors and linear transformations are represented in numerical terms 
by matrices, and how to calculate the effect on these matrices of changing 
the bases. Second (in the rest of the unit), it builds up a systematic tech- 
nique for calculating the solutions of linear problems. (By a linear problem, 
we mean one of the form 


“find all ë in V such that o(€) = a” 


where o is a linear transformation of a vector space V and « is a given 
vector in its codomain.) 


Before we can even express a linear problem satisfactorily, we need to be 
able to bring our vectors and transformations down to earth, by selecting 
bases of the vector spaces and expressing the vectors and transformations 
in terms of matrices with respect to these bases. You saw how this could 
be done in Unit 2, Linear Transformations. That unit did not, however, 
discuss the problem of deciding which bases to use. The matrices are likely 
to be simpler in form for some bases than for others, and we would like to 
know how to select the ‘‘ best” bases with respect to which the problem can 
be expressed. 


This means that we are likely to want to change bases fairly frequently, 
and it would be desirable to know what happens to a matrix representing 
a linear transformation when we do change bases. Also, can we describe 
basis changes themselves in terms of matrices? 


In Section 3.1, we obtain satisfactory answers to all these questions. If U 
is an n-dimensional vector space, then we can describe a change from an 
“old” basis {w,,...,¢,} to a “new” basis {o,...,a,} in terms of an 
n x n transition matrix P, whose columns are just the components of the 
new basis vectors in terms of the old. Then if X is a one-column matrix 
describing the components of a fixed vector č in terms of the “old ” basis, 
while X’ describes č in terms of the “new” basis, we find that X and X’ 
are related by the matrix equation 


X =Px' (1) 


We also find matrix equations to describe what happens to the matrices 
of linear transformations. Suppose g: U ——> Visa linear transformation, 
and its matrix with respect to “old ” bases in U and V is A. Then we find 
that if we change the basis in U by a transition matrix P, we get a new 
matrix for ø, namely 


A! = AP. (2) 


On the other hand, if we change the basis in V by a transition matrix Q, 
we get, for the new matrix for o: 


A"=Q7A (3) 
and if we change both bases at once, we get the matrix 
A” =Q"'AP. (4) 


Furthermore, if we allow ourselves the liberty to change both bases in this 
way, then we can get a very simple form for A”, namely 


100750070 
010:+-00--:0 
00:+-010---0 
Oe he Oe a8. 
0 0 


in which the “diagonal” entries @,,,@22,-.-, @pp ate ls (where p is the 
rank of the transformation), and all other elements are Os. 


However, for the purpose of solving linear problems, it turns out that the 
appropriate thing to do is to change the basis of the codomain only, i.e. 
to multiply the matrix of the problem on the left by another (non-singular) 
matrix. To see this, let us look at linear problems in more detail. 


The problem of solving a system of linear equations such as 
2x, — x% =4 
x,—-x2=5 
is a linear problem; so is the problem of solving the differential equation 
X"(t) + X(t) = cos pt (te R). 
In the present unit we shall be concerned only with the case where the 
domain and codomain have finite dimension, so that o has a matrix 


representation; that is, with problems like the first of the above examples, 
which can be written in the matrix form 


AX =/Y, 


rather than the second, which cannot. If the matrix A is non-singular and 
we know A™!, this equation can be solved by pre-multiplying (i.e. multi- 
plying on the left) by A~', to obtain the solution in the form 


X=A7'Y. 


For example, in the system of equations above we have 


x= [=], a= zil. r=- [5]. 


Thus A7! = i 


El-bil- [E 


In practice, we may not know A7', but the same result is achieved by 
manipulating the equations by a succession of elementary operations (such 
as adding a multiple of one equation to another), whose net effect is vir- 
tually the same as multiplying on the left by A~?. This is the Gauss 
elimination method, described in Unit M100 26, Linear Algebra III. 


z3} and the solution is 


A difficulty which we touched upon in the Foundation Course is that A 
may not have an inverse, as in the following system of equations in which 
there are more variables than equations. 


2x, —X_2+%x3=4 
X =x,- x; = 5. 


For such equations, no amount of manipulation will give us a unique solu- 
tion; either there is no solution at all, or there is an infinite set of solutions. 
(In the above example, the solution set is infinite.) 


What we would like, therefore, is a generalization of the Gauss elimination 
method which will deal with any system of equations, whether its matrix 
has an inverse or not; and which will in all cases lead us to the solution 
set, whether it consists of a unique solution, is empty, or is infinite. 


The method is just like the Gauss elimination method, in that we manipulate 
the equations by elementary operations. The net effect of these operations 
(as we shall see) is to multiply the original equation AX =Y on the 
left by a matrix which may or may not be A~!. We call it Q`! instead; 
thus the final form of the equations is 


(Q"'A)X = QY, 


and we choose Q`! to make the matrix Q714 as “simple” as we can, if 
possible, the unit matrix. This simplest possible matrix is called the Hermite 
normal form of the matrix A and is the main object of study in this unit. 
The usefulness of Hermite normal form is not confined to the solution of 
linear equations: it can also be used to find the inverse mapping of any 
linear transformation, to find the linear relations that exist among any set 
of vectors, and to find a basis for a subspace of a vector space. 


Since going from the matrix A to the matrix Q~'A is equivalent to chang- 
ing the codomain basis by a transition matrix Q, it follows that the problem 
of defining Hermite normal form turns out to be equivalent to the problem 
of choosing the most convenient basis in the codomain. In the second 
section of this unit we show how to define such a basis, and use it to specify 
the Hermite normal form completely. We also use Hermite normal form 
to prove that the row and column ranks of a matrix are equal. 


In the later sections of the unit we show how to calculate Hermite normal 
forms and consider various applications: techniques for inverting matrices, 
solving linear problems, and finding linear relations among sets of vectors. 


A computer program for finding the Hermite normal form of a matrix 
and its inverse is provided in the supplementary material. It may be used 
for some of the exercises in Section 3.3. 


3.1 CHANGE OF BASIS 
3.1.0 Introduction 


This section contains the theory underlying a great deal of the manipula- 
tions with vectors and matrices that we shall do subsequently. This material 
is central to what follows in this and subsequent units; consequently, it is 
important. It is probable that you may find this section confusing at first, 
but we expect that after you have used this material it will seem almost 
obvious to you. 


We know that there are certain statements in linear algebra which do not 
depend on the choice of basis, such as whether a linear transformation is an 
isomorphism; but, there are also statements which do depend on the choice 
of basis, such as what the entries are in the matrix representing a linear 
transformation. In this section we examine what happens to statements of 
this second kind, when the basis is changed. Once we know this, we can 
change bases to our advantage in calculations, because in certain bases, 
n-tuples and matrices may take particularly simple forms. For example, 
the law of matrix multiplication is particularly easy to use when the matrix 
is diagonal: 


7 0 0 + 0 0 
the inverse of |0 12 0 is 0 0 
0 0 8 0 0 ¢ 


It will therefore be useful (if possible) to be able to convert the matrix 
representing a linear transformation with respect to one basis into a dia- 
gonal matrix representing the same linear transformation with respect to 
some other basis. 


Another example, often used in scientific applications, is the rotation of 
Cartesian coordinate systems. It is often useful (and sometimes necessary), 
to be able to describe the same vector in different coordinate systems. For 
example, suppose that we were trying to describe the bulk motion of a 
fluid, enclosed in a bucket which is rotating at a constant angular fre- 
quency, w. 


Let S be the coordinate system of an observer at rest who will see the fluid 
moving around the bucket, with a combination of the bucket’s rotatory 
motion and various other types of motion (waves, etc.). Let S,, be a coor- 
dinate system which is rotating with the bucket. 


An observer attached to the rotating system S„ would see a very simple 
state of motion of the fluid therefore, and could readily “do the physics” 
for the problem. But to find the solution to the original problem, our 
S,-observer’s results must be transformed to results valid in S. Thus, we 
must know how vectors transform from S to Sa. This is just a rotation of 
angle 0 = wt (t is time), and is a change of basis of a special sort. 


The key to the technique of changing bases is a matrix, called the matrix 
of transition for the basis change being considered; we shall find that its 
columns are the coordinates of the new basis vectors with respect to the old 
basis. Using the appropriate matrix of transition, we can go from the 
coordinates of any given vector with respect to one basis to its coordinates 
with respect to another by a single matrix multiplication. In the case of 
linear transformations the situation is a little more complicated, because 
we can change the bases in the domain and in the codomain at the same 
time, but these basis changes can still be carried out very conveniently 
using the appropriate matrices of transition. In fact, these changes may be 
viewed as arising out of two matrices of transition applied simultaneously. 


3.1.1 The Matrix of Transition 


As an example to begin with, suppose that the vector space U is two- 
dimensional. Call {a,, «2} the old basis for U and suppose that new basis 
vectors for U are g}, a5 given by 


a =a, +a 


a, = 2a, — a2 


a; 


ay az ay 


old bass: ————> application of equations > new basis 


Note that {«;, a} does indeed constitute a basis for U if {a,, @2} does. As 

we have seen in the previous unit (Theorem 1.17, page N34), there exists 

a unique linear transformation that maps a, to a, and a, to a}. For 

Theorem 1.17 tells us that if A = {«,, &«2} and B = {a',, #5}, then: 
(oe) = oy = Oy +a 

s (1) 
O(a2) = a3 = 20, — a2 

completely defines the (change of basis) linear transformation ø. This 

transformation can be represented by a matrix in the usual way, with 

respect to any basis in the domain and any basis in the codomain. Since 

the domain and codomain, in this case, are the same space U, it is natural 

to use the same basis for both; if we take this basis to be {«,, «2} then, as 

we saw in the previous unit, the columns of the matrix are the coordinates 

of the images of œ, and «, with respect to this basis. Applying Equation 

(2.2) on page N38, with A = B = {a,, a}, we have: 
o(a) = 41%; + 424% 

(2) 
o(a) = 442% + App ty 

By comparing Equations (1) and (2), we see that the matrix representing 

the transformation from the new basis to the old, with respect to the old 


basis, is 
1 2 
1 -If 


This is called the matrix of transition* for this particular change of basis. 
To single out its particular use, we use the symbol P for a transition matrix, 
and pj; for its entries. If there are two transition matrices being used in the 
same problem, we shall use the symbols P and Q. 


The corresponding formula for a general change of basis is given in the 
next reading passage. 


READ from the top of page N50 to the sentence “Thus P is non-singular.” 
about half-way down the page. 


Note 


Just as in Equation (2.2) on page N38, so in Equation (4.1) on page N50, we sum 
over the first of the double suffices, the one labelling the rows. This is the way it is 
always done in an equation involving matrices and vectors. 


* We see how this matrix relates the coordinates of a vector with respect to the new and 
old bases shortly. 


Exercises 


1. Let U = R?, a, = (1, 0,0), a = (0, 1, 0), «3 = (0,0, 1), a = a, 1,1), 
a, = (2,1, 1) and a =(1, — 1, 0). Find the matrix of transition with 
respect to the old basis {a,, &2, &3}- 

2. Exercise 1, page N52. 

Solutions 


10 


1. We must start by finding out just what particular linear 
combination of {a,, «2, a3} each of the aj is. Direct inspec- 
tion shows us that 

a =a, +a, +43 

u, = 2a, + az + 43 

a3 =O, — a. 
Now 


ot, = (a) = P11% + P2182 + P31%3, etc. gives 


1 2 1 
P=j1 1 —1}]. 
1 1 0. 


2. Writing «, =1, w, =x, «3; =x”, the new basis ıs given in 
terms of the old by 
a =p (=x t+x+1l=a,+a, +03 
a = p(x) =x? — x — 2 = -2a, — 4 +03 
a = p(x) =x? +x — 1 = a, + az +05 


1 -2 -1 
and so the matrix of transition is f -1 i 
1 1 1 


3.1.2 Change of Basis for Coordinates of Vectors 


As our first application of the matrix of transition we consider the problem 
of finding the coordinates of vectors with respect to one basis when the 
coordinates with respect to another basis are given. Loosely speaking, 
since a vector can be written as a,a, +a, +, where the a, are the 
coordinates and the a, the basis vectors, if the basis vectors change in a 
certain way, the coordinates must change ina related way in such a fashion 
that the vector itself does not change. This is the crux of the matter. For, the 
vector exists as an entity in its own right (think of a geometric vector in 
space); the basis vectors are chosen independently of the existence of the 
vector in question. Let us use our example from sub-section 3.1.1 to illus- 
trate the problem: we have calculated already the matrix of transition. Thus, 
{a;, æ2} is the old basis, {a, = a, + %2, a5 = 2a, — æ} is the new basis of U, 
and P = [i 4 is the relevant matrix of transition. Let č be a vector 


in U: 


old basis; vector & new basis ; same vector č 


Observe that & is the same in both cases. Once this point is grasped, the 
rest is arithmetic. 


The coordinates of č, as opposed to č itself, do depend upon the basis 
chosen: they measure the “ geometrical projection” of € on to the basis 
vectors. 


Now 

Č = X8, + X203; (la) 
but as well, 

č = xia; + x202. (1b) 


Because the «; are (known) linear combinations of {«,, a2}, the x; are related 
linear combinations (to be found) of {x,, x2}. Setting č = č with the 


left-hand side obtained from Equation (1a) and the right-hand side obtained 
from Equation (1b), we have 
Xa, + XQ a, = X10] + Za 
=x; + a2) + x3 (20, — a2) 
(x1 + 2x3); + (x1 — X4)m2- 


Linear independence of a, and a, now gives us 
x =x, + 2x5 Q) 


r , 
x =X, — x4 


These equations can be written in matrix form 


EI-t -AE 
kJ- i 


where P = $ a is just the matrix of transition for this problem. 


or 


Notice that the equation gives the old coordinates in terms of the new 
ones, whereas in the preceding section we expressed the new basis vectors 
in terms of the old ones. The two concepts are “complementary ”. If we 
have the new basis vectors in terms of the old, then the natural substitution 
in any expression, is to replace the new basis vectors by the old. We then 
inevitably get the old coordinates in terms of the new ones. Look at the 
calculation we performed above, and make sure you see why this happens 
before reading the next piece of N. Note that Equations (2) or (2a) are a 
system of linear equations relating the coordinates, and that the matrix 
of coefficients is P. That is, we can write Equation (2a) in indexed form as 


x=} pyx. 
dJ 
Here the summation is over the second index not the first as in sub-section 


3.1.1. This is another facet of the complementary nature of basis vectors 
and coordinates. 


READ the rest of page N50. 


Exercise 

Use Equation (4.3) on page N50 to express the vector 
č = —2a, + a — 305 in P3, 

in terms of {a, = 1, a, = x, a3 = x°}, where 


aaxttxt+ log xaa. 


Check by direct manipulation of the polynomials. 


Solution 


- 1 -2 -1 
eevee x= | i and a -1 es 


=3 1 1 1 


the previous solution. Hence Equation (4.3) gives 


a 


by matrix multiplication. 


12 


That is, č = — a, — 6a, — 4a. 


Check € = =x? +x +41) + (x? — x —2)— 3(x? + x — 1) 
= —4x? ~6x—1 
= —a, — 6a, — 4a; 

as before. 


3.1.3 Change of Basis in the Domain of a Linear 
Transformation 


If ø is a linear transformation with domain U and codomain V, the matrix 
representing o will change if we change the basis in either the domain or 
the codomain or in both. This is a matter of definition. The matrix entries 
a;j in Equation (2.2) on page N38 


m 


ola;) = py ai; B; 


clearly depend upon the basis vectors {a}, {B;}. 


It is less complicated to consider a basis change in the domain only first, 
then a basis change in the codomain only, and finally put them together 
to form the general case. 


The result will be concisely stated in matrix form, but deriving the result 
involves a certain amount of index manipulation. To help with this 
difficulty, we have devised an example. Therefore let us consider the case 
where U is 2-dimensional, with a basis A = {a,, 42} and V is 3-dimensional, 
with a basis B={f,, 82 B3}. We know from an earlier section (Theorem 
1.17, page N34) that a linear transformation is completely determined by 
what it does to a basis in the domain. A general linear transformation 
ao: U —— V can therefore be specified by giving the images of a, and 
&. Suppose these are 


a(x) = By + 12 Bo + r3 Bs 
alaz) = 5,8; + 5282 + S3 B3 


where r,, 2,13, 51,52, 53 are arbitrary scalars. 


a) 


Let M be the matrix corresponding to o with bases A in U, and B in V; 
then the columns of M are the coordinates of o(«,) and o(«,) with respect 
to the basis B; so Equation (1) gives 


ri sS 
M=|r, J $ 
Fa. S3 
We already have a particular example of a basis change in U worked out in 
the preceding two sub-sections, where we change the basis in U from {a,, 2} 
to {o, a5} with 


(2) 


The matrix of transition was 


-iA o 


Now {a, a} is also a basis for U, and we can also find the matrix, M’ 
representing the same linear transformation, ø, with respect to this new 
basis for the domain, {a,«,}, and the same basis for the codomain 
{B,, B2, B3}; this is what this sub-section is all about. First note that g can 
be thought of as having an existence independent of bases, just as a vector 
does. It is only the representing matrix which depends upon bases. 


Having understood this, the rest is again tedious arithmetic. It is also clear 
that it will not be accidental if.’ is related to M by the transition matrix 
P. Intuitively, the entire transformation process is contained in P, and every 
change which depends only upon the change of basis must somehow 
depend only upon P; it is the only “ variable” in the problem. Now to the 
arithmetic. We follow a “ programmed” procedure, inviting you to supply 
the missing information. 


Exercise 


To find M’ we apply Equation (2.2) on page N38 again. This requires 
knowing the o(«;) as lincar combinations of the basis vectors 


@ 


of V. While we do not know this directly, we do know the images of the 
original basis vectors 


(ii) 


as linear combinations of {8,, 82, B3}, and also the new basis of U 


(iii) 
as linear combinations of «, and «z. Putting these together as follows 
yields M’: 


O(a) = ofa, + a2) 


-C e a a 


and 
ala3) = o(2a, — a2) 
=[ a+ Bil Je w 
Thus 
M'= | | (vi) 
Solution 
(i) {Bi Ba» Bs} (i) o(@,), olea) Gii) {a1 2} 
(iv) (ri + s1), (2 + 52), (rg + 53) 
(v) (2r, — s1), (2r2 — s2), (2r; — 53) 
(vi) [rr ts, 2ry— sy 
rg +S 2r- s 
r3 +53 2r3—53 
Exercise 


With M, P and M’ as in the text above, compare MP with M’. 
Solution 
ry Sy fi ‘| Tr +s 2ry—s, 
MP= Ir, s||1 -1]= tg +s 2rg—s,} =M’ 
rz) S3 F3 +S, 2r3— 53 
The generalization of this result is given in the next reading passage. 


READ page N51 to“... we have A’ = AP” (about half-way down the page). 


14 


Note 


Equation (4.6): In this equation, Y = AX is the matrix representation of the 
action of o, from Equation (2.9) on page N41. The next step uses Equation (4.3), 
and the last uses the associativity of matrix multiplication. Notice that the matrix 
of transition appears at the right of the product, AP. 


3.1.4 General Changes of Bases for Linear Transformations 


The next reading passage shows the same kind of calculation applied to 
changes of basis in the codomain instead of the domain. If you find the 
proliferating suffixes hard to interpret, especially in Equation (4.7), con- 
struct an example for yourself similar to the one at the start of the previous 
sub-section. For instance, use the previous example with {æ}, a2} a basis for 
U, o as in Equation (1) of sub-section 3.1.3, {8;, B2, B3} the old basis for V 
and {B4, 83, B3} a new basis for V, with 


Bi = By -b 
B2 = 28, — Bs 
Bs = Bi + P3. 


Find the transition matrix for {B{, 22, B3} to {B,, B2, B3}, and call it 
Q7'. (Note that Q is the transition matrix for {8,} to {B/}.) Then find the 
matrix M” representing o with respect to {«,, «2} and {8}, 82, 83}. Then you 
are on your own with Nering for the next reading passage. 


READ page N51 from “ Now consider the effect...” to“... cis Q~'AP”. 


Note 


line —4 page N51 To derive the formula Q-'AP, think of the basis changes 
being made in succession. We start with the matrix A with respect to the original 
basis. A change of basis in the domain, with fixed codomain basis, changes the 
matrix to AP (Equation (4.5)). Then we change the basis in the codomain, and 
Equation (4.8) applied to the current matrix, which is AP rather than A, tells us 
that the final matrix is Q-'AP. 


You may find the following interpretation helpful. In a product of linear 
transformations, say ot, we do the right-hand one first, since ot(«) = 
o(t(«)). Similarly for matrices, a product Q~'AP, which represents a linear 
transformation, can be interpreted as doing P first, then A, then Q™'. By 
Equation (4.3), the matrix P takes us from the new coordinates to the old 
in the domain. Then A takes us from these to the codomain (using the old 
coordinates in both spaces). Finally, by Equation (4.3) again, to get from 
the old coordinates to the new in the codomain we use Q~! (since Q would 
take us from the new to the old). 


domain codomain 


t Pla) + Q(B) 
old basis 3 old basis 
new coordinates to old coordinates to 
p old coordinates now coordinates 

Ww 

-f 

4 1 
a aY 


new basis new basis 


15 


Exercises 


1. For the example outlined: in the first paragraph of this sub-section, 
find Q~' and the matrix M”. Verify that M” = Q7'M. 

2. (Continued) Find the matrix M” representing o with respect to 
{a =a, +a, %3 = 2a, —a} and {f, 83, B3} directly. Verify that 


M"= 


Solutions 


16 


Q-'MP. 
1 0 1 
Q'=|-1 2 #0 
0 -i 1 
Now 


ala) = 1,8, +r2b3 + rapa 

=r (b; — B2) + r2(283 — BS) + r3(81 + B3) 

= (r +r3)bi + (ri + 2ra)b3 + (— t2 + 13) 85 
o(a) = 5,8, + 52 Bz +5383 

=(s; + 53), + (—5; + 252)B + (—52 + 53)B5 


so that 
ri +r 5, +53 
M"=|-r,+2r. -—5, +25, 
=r +r; -sts 
Also 


r +r Sy +53 
= /—ry+2r, —s,+2s, 
=r: +r =s +83 
In sub-section 3.1.3 (preceding the exercise) we found that 
alai) = (ry + 31)Bı + (r2 + 52)B2 + (r3 + 53)B3 
whence 
olai) = (ry + s1)X(81 — B2) + (r2 + 52)(2B5 — 23) 
+ (r3 + 53)(Bi + 83) 
= (r; +5, + r3 + 53)8, + (—ry — s, + 2r, + 2s3)8, 
+(=rz — 52 + r3 + s3)b3 
Similarly , 
O(a) = (2r; — s1)8, + (2r2 — s2)f2 + (2r3 — s3)B3 
= (2r, — s1)(B1 — B2) + (2r2 — 52)(2B4 — B3) 
+ (2r3 ~ 53)(B1 + 83) 
= (2r; — sı + 2r3 — 53)B, + (—2r, +5, + 4r, — 252) 
+(—2r + 52 + 2r3 — 55)B5 


so that 
Fi ts, try + S3 2r, = s, +2r3 — 53 
M" = | =r, — s, + 2r3 + 2s, —2r, +s, + 4r, — 2s, 
r2 S2 +13 + S3 = 2r + s2 + 2r3 — s3 
=Q7'MP. 


3.1.5 The Simplest Matrix for a Linear Transformation 


By choosing suitable bases in both domain and codomain, we can bring 
the matrix representing the transformation to the very simple form shown 
on page N52. The secret, loosely speaking, is to choose a basis in the 
domain which has a proper subset which is a basis for the kernel of ø, 
and to choose a basis in the codomain which includes all the non-zero 
images of domain basis vectors. 


Let us explain this! If the rank of ø is p and U is of dimension n, then K(o), 
the kernel of ø, is of dimension 7 — p, since by the dimension theorem, 
n — p = v, the nullity of ø. We can choose a basis of n — p = v vectors 
for K(ø) and then complete it to obtain a basis for U. Let the basis so 
obtained be 


basis for U 
a 
Ea (ge , 
Qis A2» es hy Ca ty ees Oy 


basis for K(o) 


Notice that we have labelled the basis vectors of K(c) so that they come 
last. 


As usual we write 


But a, E€ K(a), so that o(a)) = 0 (the zero vector) 
ie. 

Ain = Aan = = Ayn = 0 
which tells us that each element of the nth column of the matrix A’ repre- 
senting ø is zero, Similarly, all the elements of the last v columns of A’ 
are zero. Notice that this is so whatever the choice of basis vectors f; 


in the codomain. If {a,, &2, ..., æ} was the “old” basis for U with corre- 
sponding matrix A, then we can interpret our result so far, as showing that: 


If A is any m x n matrix of rank p, there always exists a non- 
singular n x n matrix P such that 


A’ = AP 
has the last » — p = v columns all zero. 


This is the first step. The next step is to choose a “new” basis in the 
codomain in order to make a further simplification. We again use the 
dimension theorem; this time a step we noted in its proof: instead of 
By,.-+, By we use B3, -> Bp, where 


B= o(a;) (i=1,...,p) 
and complete to form a new basis. 


READ the remainder of Section 4, pages N51-52. 


Exercise 


Complete the following statements to describe the steps in the proof of 
Theorem 4.1. We suggest you try to complete these steps without reference 
to our text or N. 


(i) Given any m x n matrix A, we can regard it as representing a linear 
transformation ø of an n-dimensional space U to an m-dimensional 
space V with respect to given bases {a,,...,¢,} and {f,,..., Bm} 
respectively. 
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(ii) We first choose a new basis {a}, ..., an} for U in the following way: 


We denote the transition matrix for this change of basis by P. 


(iii) We then choose a new basis {6}, ..., Ba} for V in the following way: 


We denote the transition matrix for this change of basis by Q. 


(iv) The matrix which represents g with respect to the new bases in U 
and V is Q~'AP, and its elements can be described as follows: 


Solution 


(ii) If v=—~p is the nullity of ø, then we choose the new 


basis, {a,..., a}, So that {a 4), @42,--.,a,} is a basis 
for K(o). 
Gii) Bj = o(aj) i=1,..., p; olx) =0, i=p+1,...,m, so we 


choose f,4,,--., Bm in any way that gives a basis B’ of V. 
(iv) p columns y columns 

100+ 0} 
010: 0} 
{ 

prows}> > + ct a | 0 
i] 

m — p TOWS 0 0 


3.1.6 Summary of Section 3.1 


In this section we defined the terms 


matrix of transition (page N50) 


k*k 
similar (page N52) txs 
Theorem 
(4.1, page N52) 
If A is any m x n matrix of rank p, there exist a non-singular n x n matrix * * 
P and a non-singular m x m matrix Q such that A’ = Q7'AP has the first 
p elements of the main diagonal equal to 1, and all other elements equal to 
zero. 
Techniques 
1. Determine the matrix of transition for a given change of basis. ke 
2. Determine the coordinates of a vector with Tespect to a new basis, tr 
when given the coordinates of that vector with respect to an old basis 
and the appropriate matrix of transition. 
3. Determine the matrix of a linear transformation, ø, from U to V with tx: 


respect to bases 4’ in U and B’ in V, given the matrix Tepresenting o 
with respect to bases A in U and B in V, and given the form of the 
change of bases A to A’ in U and B to B' in V. 
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3.2 WHAT IS HERMITE NORMAL FORM? 
3.2.1 Choosing a Basis in the Codomain 


In the previous section we have seen that the matrix representing a linear 
transformation can be greatly simplified by a judicious choice of bases 
in the domain and codomain. Some simplification can still be achieved 
if we restrict ourselves to a basis change in either the domain or the codo- 
main alone. 


We stated in the Introduction, 3.0, that, from the point of view of solving 


systems of linear equations, basis changes in the codomain are particularly 
important. 


For a given basis {a,,...,a,} of the domain, the choice is very simple 
if the transformation is an isomorphism, for then {o(a,),..., o(@,)} 
forms a basis of the codomain and we simply take this as the new basis 
{Bi,---, Bj}. (The primes over the fs indicate, as usual, that this may 
not have been the basis with respect to which o was expressed originally; 
this would have been written {f,,..., 8,}.) Our definition for the new 
basis implies 


alai) = By, ala2) = By, ..., o(@,) = Br, 
so that the matrix of ø, with respect to these bases, is just the n x n unit 
matrix, 7. The reason why such a simple matrix representation is possible 
here is that ø, being an isomorphism, has a non-singular matrix A. The 
change of basis we have made corresponds to using A as the matrix of 
transition in the codomain, so that the transformed matrix is not Q~'A 
but A~!4 = I (by Equation (4.8), page N51). 


Example 1 


The following diagram illustrates the situation in the case where U and V 
are 2-dimensional, and o(«,) and o(«) are linearly independent, and can 
therefore form a codomain basis {f{, 3}. We have drawn the diagram 
to illustrate the particular case where 


o(a;) = 38, + 482 = fi, 
alaa) = Bı + 2B. = By. 


The matrices representing o with respect to the original codomain basis 
{B,, B2} and the new codomain basis {£;, £3} are, respectively, 


kal bil 


(Remember that the columns of the matrix are the coordinates of the 
images of the domain basis with respect to the appropriate codomain 
basis. Also, the original bases {a,, x2}, {8,, 82} are drawn at right-angles 
simply to tidy up the diagram and make things easier to visualize. There 
is no mathematical significance in their being at right-angles.) 


U v 


o(a,)=B,’ 


a 


A more interesting situation arises if the images of the domain basis 
elements are linearly independent, but do not span the codomain. 
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Example 2 


This example is exactly the same as the previous one, except that V is 
3-dimensional. so that there is an extra codomain basis vector f}. but we 
still have 


aola) = 38, + 4p, 
ola) = B, + 2B. 
The matrix of ø with respect to {x,, #2} and {B,, B2, B3} is now 


3 1l 
A=]4 2j. 
0 0 


The images of the domain basis are linearly independent, just as in Example 
1, but this time they do not span FV. It is clear, though, that if we take 


By = ala), B3 = ala), BS = Bs, 


then the matrix for ø is now 


0 0 


which is as simple as we can hope to make it. Furthermore, we get the 
same matrix whatever we take f3 to be (consistent with the requirement 
that (8), 22, B5} forms a basis; i.e. 83 is linearly independent of 8‘ and £5). 
The choice of £5 does not affect the fact that the images of a, and a, are 
expressed entirely in terms of 8} and £}. If you go back to the previous 
figure, and consider V to be 3-dimensional, i.e., some vectors can come 
out of the page at you, then you can ‘imagine f4 to be any vector that does 
not lie in the plane of the page. You can then see that the one you choose 
for f; does not have any effect on the relations between vectors that do 
lie in the plane of the page. 


We shall use just the same idea in the general case. Suppose we have a 
transformation from an n-dimensional space U to an m-dimensional 
space V. In general, o(a,), ..., o(a) may fail to be linearly independent, 
and may also fail to span V. Suppose Lala), ..., o(@,)> is k-dimensional; 
then we shall choose our first k basis elements Bi, <--> By in a certain way, 
to lie in <o(a,),..., o(a,)>, so that all of o(a,),..., o(a,) are expressible 
as linear combinations of these. Then we can complete to a basis {f,..., 
Bu} of V in any way we like, in the confidence that no matter how we 
choose Biar., Bm, we have completely specified the form of the matrix 
by our choice. of the first k basis elements. In fact, if all we want to do is to 
find the Hermite normal form, we do not even bother to calculate suitable 
Bitty +++, Bi Theorem 3.6 on page N17 tells us that we could do so if we 
felt like it, and this is enough. 


Example 3 


Let U = R?, V = R?, and {«,, >}, {B,, B2} be the standard bases.* Consider 
the transformation c: U —— V whose matrix with respect to these 


bases is 
2 -4), 
3 -6|’ 


then o(a,) = (2, 3), o(a2) = (—4, —6), and these are linearly dependent, 
so they cannot both be elements of a basis for V. 
* The standard basis for R? is {(1, 0), (0, 1)}. 

Likewise the standard basis for R? is {(1, 0, 0), (0, 1, 0), (0, 0, 1)}, and so on. 
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But let us take f' to be o(a,), at any rate. Do you see what has happened 
now? Both o(a,) and o(a2) are now expressible in terms of ĝi, and once 
more we have completely specified the matrix. No matter how we choose 
B2, we now have the matrix in the form 


b a 


a 


o(az)= By 


The whole of U is mapped to the subspace of V spanned by f, = 28, + 382. 
Since o(«,) is always dependent on o(a,), no matter how we choose f; 


and ĝ;,, it would seem that we cannot make the matrix any simpler than 
the above by altering the codomain basis alone. 


Example 4 
This time let us have U = V = R?, and a transformation ø whose matrix 


with respect to the standard bases (for both U and V) {(1, 0, 0), (0, 1, 0), 
(0, 0, 1)} is 


1 1 2 
A=; 2 -1 1}. 
=1 2 1 


Jn this case the last column is the sum of the first two, so that 
(a3) = o(a) + o(@2). 


That is to say, o(a,), o(«2) and o(a3) are linearly dependent, but any two 
of o(%,), a(x2), o(a3) are linearly independent. So it would seem that we 
should choose two of them as basis vectors, then make an arbitrary choice 
of the third basis vector. But which two do we choose? 

It doesn’t really matter exactly what rule we adopt at this point, but we 
must have some definite rule, and then stick to it in all cases, if we are to 
end up with a unique specification for Hermite normal form. In fact, the 
rule which is chosen stipulates, in the case of Example 4, that o(a,) and 
o(a) are to be our first two basis vectors, and then we can make an arbi- 
trary (linearly independent) choice for the third vector. 


Exercise 


Write down the matrix of the transformation of Example 4, when the 
codomain basis is: 


(i) By = ola), B3 = ale), B3 = Q, 1, 5); 
(ii) By = ola), B3 = alo), B = (5, 1, 2). 
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Solution 


Since a(a3) = o(a,) + o(a2), the matrix is 


101 
ol 1 
000 


in the case of both (i) and (ii); again, the linear relationships 
determining the matrix are unaffected by the choice of 3. 
Exercise 
(Quite difficult, but do not spend more than a few minutes on it.) 


Suggest a rule for choosing the first p elements of the codomain basis, 
when 


Im(o) = Co(e,), ..., o(&,)> 
is p-dimensional, but o(,), ..., a(«,) are not linearly independent. 
The solution to this exercise is contained in the text that follows. 
First of all, let us look at an example where <o(«,), ..., o(«,)> is 3-dimen- 
sional, but o(«,), o(a2), (3) are not linearly independent. 
Example 5 


Let U = V = R$, and let g: U —— V have the following matrix with 
respect to the standard bases: 


1233 
2134 
4-130 35 
2136 


so that 
o(a,) = (1, 2, 3, 2) = first column of A, 
o(a) = (2, 1, 0, 1) = second column of A, 
and so on. 
Now (2, 1,0, 1) is not a multiple of (1, 2, 3, 2), so that o(a,) and o(a) 


are linearly independent, and can presumably be taken to form the first 
two elements of the codomain basis for the Hermite normal form, A’: 


By = o(a4) = (1, 2, 3, 2) (1) 
By = o(@2) = (2, 1,0, 1) (2) 
Now this immediately tells us what the first two columns of the new matrix 
are going to be. These columns, after all, are the components of o(a,) 


and ø (#2) with respect to the new codomain basis; and however we choose 
p3 and $4, Equations (1) and (2) can be rewritten: 


ala) = 18, + OB, +083 + 084 (la) 

ala2) = Of; + 183 + OBS + 02, (2a) 
giving, for the first two columns of the new matrix A’: 

1 0 

0 1 

0 0 

0 0 


What about o(«3) = (3, 3, 3, 3)? It is not difficult to see that we have the 
linear relation 


G, 3, 3, 3) = (1, 2, 3, 2) + (2, 1, 0, 1) 
= o(«,) + ofa). (3) 


22 


We cannot therefore use (3, 3, 3, 3) in building up the codomain basis. 
In fact, Equation (3) tells us that, having fixed f, and $}, we have auto- 
matically specified what the third column of A’ must be: 


a(x) = 18, + 185 + 085 + 083, (3a) 
so the third column must be 


1 
1 
0 
0 


Now what about o(«4) = (3, 4, 5, 6)? If this also turns out to be dependent* 
on a(,) and o(«,), then we have specified the fourth column of A’, and 
need look no further. However, we are not so lucky this time. Note that 
o(a,) and (a) each have the second component equal to the fourth 
component: 


l Lod 
o(%,) is (1,2,3,2) and ø(@)= (2, 1, 0, 1). 


So this must be true of any linear combination of o(«,) and o(a). But 
it is not true of o(a,4): for, 4 #6. Thus o(a,) is independent of o(«,) and 
o(a2), and can be taken to be a codomain basis element, 3. 


B3 = ala4), ` (4) 
alaa) = OB; + O85 + 185 + 084; (4a) 
and so the fourth column of A’ is 


0 
0 
l 
0 


and we have found A’ without having to specify the fourth codomain 
basis clement £4. This can be chosen arbitrarily; but since we have already 
found 4’, there is, in practice, no need to do so. A’ is now equal to 


1010 
0110 
0001 
0000 


It should now be apparent what the general rule must be for choosing 
the first p codomain basis elements out of {o(a,),..-,o(a,)} (where 
<a(a;), ..-, a(&,)> is p-dimensional). In fact, we choose them in the same 
way as in the proof of Theorem 3.5, page N16, which you met in Unit 1, 
Vector Spaces: go through the o(«,) in order, picking out those which are 
independent of the previous ones, and rejecting those which are not. This 
gives us a systematic method of choosing the required codomain basis 
vectors so that they and the order in which they appear are exactly speci- 
fied. We can give detailed expression to this rule as follows. 


Go through each of the o(«,) in turn, starting with o(a,), then o(a,), etc., 
and accept it as a member of the basis if it is independent of all the pre- 
vious ones, but reject it if it depends on the previous ones. Thus £; is the 
first non-zero element of {a(a,), ..., o(@,)}; B is the first element of 
{o(a,), ..., o(a,)} not depending on £4; 23 is the first one not depending on 
p; and f; and so on. The number of vectors we obtain in this way must be 
p, the rank of ø: it cannot be more, since the selection process ensures that 
the vectors selected are independent, and it cannot be less, since they must 
span the p-dimensional space Cola), ..., 0(%,)>. This is because we look at 
each of the a(a;) in turn, and give it the chance of contributing something to 
<o(a,), .... o(a,)>. If it adds nothing to what has.gone before, ie. if it is 
dependent on the previous o(«;)’s, then and only then do we reject it. 


* Here, of course, dependent means linearly dependent. 


23 


LM 3.2.1 


The matrix for ¢ obtained by choosing the first p elements of the codomain 
basis in this way, and the others arbitrarily, is the Hermite normal form 
we are looking for. Often, what interests us is the transition from the 
original matrix to the new matrix, so the process is usually described as 
reducing a matrix to Hermite normal form. 


There is a rather neat way of characterizing what is going on here, in terms 
of the dimensions of the spaces spanned by sets of image vectors. This 
comes about as follows. We have a fixed basis {a,, ..-, @,} in the domain, 
and at the kth step in the process of choosing a basis in the codomain, we 
look at the first k elements «,,..., a, of the domain basis, and at their 
images o(a,),..., o(a%,). Now, since {«,,...,@} is linearly independent, 


ay, <- -s Xk) is k-dimensional. 


But {o(a,),..-, o(%)} is not necessarily independent; what can we say 
about the dimension of <o(a,), ..., o(&%)>? 


For convenience, let 
U, = Kais os MDs 


Then <a(a,),..., o(x,)) is the image of <a,,...,0%,>, namely o(U,), and 
the problem is to find dim o(U,). 


Well, it should be clear that dim o(U,) must be equal to the number 
of elements of the codomain basis that are selected out of the set 
{o(w,), ..., o(%,)} by the selection process used above. This is because, by 
construction, they are linearly independent, and span o(U,). (The latter 
fact follows because those vectors o(«;), for i < k, that are not in this set, 
are linearly dependent on those which are.) 


Thus, every time a vector a(«;) is chosen as a basis vector in the codomain, 
the dimension of o(U;) becomes one more than the dimension of o(U;_,), 
whereas every time a vector fails to be chosen, the dimension of o(U;) 
and o(U;_,) are equal. We can look at this the other way round, and 
characterize the codomain basis vectors by saying that: 


a(«;) is a codomain basis vector if and only if 
dim o(U,) = dim o(U;-,) + 1 


We need another piece of notation, to help us to distinguish between those 
elements of {o(«,),..., 6(«,)} which are specially chosen to be codomain 
basis vectors, and those which are not. In order to do this, we give a special 
notation to the suffices corresponding to the chosen codomain basis 
vectors; they are denoted by k,,...,k,. (Remember, p is the dimension 
of o(U), and so o(U) has p basis vectors.) Thus the corresponding elements 
of the codomain basis are denoted by o(a,,),..., 6(%,) (or as Bi, ..., By, 
whichever is more convenient in the circumstances). 


For instance, in Example 5 above, k, is 1, k, is 2, and k, is 4, so that 
(a) = Bi, ala2) = Bz, and o(a4) = $3. 


Exercise 

If the above method is applied to the matrix of Example 5, namely 
1 2 
21 
3 0 
2 1 
then o(U,) is <(1, 2, 3, 2)>, and has dimension 1. 


Write down the dimensions of o(U2), o(U3), o(U4), and specify a basis for 
each. 
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Solution 


o(U,) has dimension 2 and a basis {(1, 2, 3, 2), (2, 1, 0, L}. 

o(U3) has dimension 2 and the same basis as above (since 
(3, 3, 3, 3) € o(U,)). 

o(U,) has dimension 3 and a basis 


{(1, 2, 3, 2), (2, 1, 0, 1), (3, 4, 5, 6)}- 


The discussion in this sub-section is an extensive elaboration of the first 
two paragraphs of Section 5, pages N53-54. We have not asked you to 
read this piece for two reasons: 


(i) it refers to Theorem 4.8 of Chapter I, which we have not covered; 
(ii) in testing this unit, some students found the text of N particularly 
difficult at this point. 


If you have the time you might like to look at these two paragraphs of N, 
to see how concisely the ideas can be put. It is a matter of taste (and 
possibly mathematical fashion) whether one regards such conciseness as 
having merit. Certainly, for those who understand it, it has great appeal, 
much like certain music and poetry. 


3.2.2 Recognizing Hermite Normal Form 


The method we used in the previous sub-section to bring a matrix to 
Hermite normal form can be stream-lined if we do not write out all the 
vectors considered, but work directly with the matrix. The method for 
doing this is described later in this text. To apply this method, it is necessary 
to be able to tell by looking at a matrix whether it is in Hermite normal 
form or not, and this is the subject of the present sub-section. 


In Example 5 of the previous sub-section we reduced a matrix to the form 


1010 
0110 
0001 
0000 


What features of this matrix would also be found in the Hermite normal 
form of a general matrix? 


The columns of the above matrix are of two types. Columns 1, 2 and 4 are 
the images of a,, a and «, referred to the new codomain basis. Since these 
images are themselves the codomain basis vectors, these columns are 


1 0 0 
0 1 0 
op Jo; 1 
0 0 0 


In the general case, the image space has dimension p (the rank of the 
matrix), there are p such columns, and the ith of them (which is the k;th 
column in the whole matrix, if we use the notation we have just developed) 
consists of a solitary 1 in the ith position and zeros everywhere else. 


a(x) ala) ala3) olaa) 


1 0 1 0 

0 1 1 0 

0 0 0 1 

0 0 0 0 
O(c) o(o%,,) o(%,,) 


The second type of column is exemplified by column 3 in the example. It 
gives the coordinates of o(a3), which is not included in the codomain basis 
because it is a linear combination of its predecessors. In general, every 
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column of this type is a linear combination of the columns to its left. 
Such a column can have any numbers in the positions with non-zero 
entries to the left of them, but only zeros in the positions with nothing 
but zeros to the left of them. Thus a general Hermite normal form matrix 
looks something like this: 


0 x x 
0 > 0 x x 
0: 0 ioi x x 
0. = 0 iol 0 x x 
0 o: Lo: iol foto o 
0 jo} io! io ojo 0 


where the crosses can be any numbers. (The staircase-shaped line is 
included to guide the eye only.) 


The next reading passage re-formulates the above specification in a more 
general way. 


READ from “ Since o(ap,) = B; +++” on page N54 to the end of the statement 
of Theorem 5.1 on page N55. 


Notes 


(i) First paragraph of passage. This can perhaps be put more simply as follows. 
The domain basis vectors which have been labelled a,,..., &xp, have as their 
images the codomain basis vectors 84, ..., 84. Thus their matrix representatives, 
the columns kı, ..., kp, each have a | in the appropriate place (the ith place) 
and zeros elsewhere. Next, consider a typical domain basis vector «,, with 
ky <j < kısı. Its image is a linear combination of the codomain basis elements 
encountered so far, i.e. of Bi,..., Bi. Thus the corresponding matrix representa- 
tive has Os below row i, and can have any numbers in the first i places (since the 
vector in question can be any linear combination of £4, ..., Bi). 

(ii) line —2, page N54 Here begins a 3-part criterion for recognizing matrices 
in Hermite normal form. Do not infer from the wording of part (1) of the criterion 
that it is necessary to know the rank p before we can apply the criterion. On the 
contrary, if all 3 parts of the criterion are satisfied for some value of p, then that 
value of p must be the rank. 


(ili) The last line of the statement of Theorem 5.1. This is dealt with in sub-section 
3.2.3. 


Exercise 

Do Exercise 1 on page N56, and determine the ranks of those matrices 
that are in Hermite normal form. 

Solution 


Matrices (a), (c) and (d) are in Hermite normal form, and they 
all have rank 3. 


Matrix (b) violates part (2) of the criterion on page N55, because 
the first non-zero element in row 1 is not a 1. 


Matrix (e) violates part (3) of the criterion, because (with k,=1, 
kı =2, k, = 4, k4 = 5) there is a non-zero element in column 5 
other than the 1 in row 4. 
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3.2.3 The Uniqueness of Hermite Normal Form 


Reviewing what we have done so far in Section 3.2: 


(i) We have seen that choosing a suitable codomain basis can greatly 
simplify the matrix of a transformation. 
(ii) We have seen how to lay down a rule for choosing the codomain 
basis, which uniquely specifies a form for a particular matrix. 
(iii) We have specified the conditions which a matrix obeys if it is in this 
form, namely Hermite normal form. 


What we have not done, is to prove the uniqueness of Hermite normal 
form. There might, for all we know so far, be a different codomain basis 
choice, giving a different form for the matrix, but still obeying the three 
conditions laid down in Theorem 5.1, pages N54-55, for determining 
Hermite normal form. The proof that this is not so, and that there is a 
unique matrix in Hermite normal form to be obtained from any given 
matrix by a change of codomain basis, is given in the next reading passage. 
You can consider this proof as a test for yourself; if you can follow it 
quite easily, then you have probably grasped the material so far. If you 
cannot follow the proof, then read again Section 3.2 up to this point, and 
go over the exercises again. 


READ page N55 from“ The form A’ is...” as far as but excluding Definition. 


Notes 


(i) line 19, page N55 Condition (3) tells us that the &,th column, which gives the 
image of a, in the new basis, has a 1 as its ith entry and zeros for all the others, 
so that this image is the ith codomain basis vector both in 8; and B3. 

(ii) line 20, page N55 Condition (1) tells us that every column has only zeros for 
the (p + 1)th entry onwards, so that all the images of the domain basis vectors 
are linear combinations of {o(as,),..., o(@s,)}. The coefficients in these linear 
combinations are the non-zero entries in the columns of the matrix, which are 
therefore the same both for B; and B3. 


Exercise 
If A, B and M are matrices, with M non-singular, such that A = MB, what 
can be deduced about the Hermite normal forms of A and B? 
Solution 
The Hermite normal forms of A and B are identical. 


(Proof Theorem 5.1 tells us that there is a non-singular matrix 
Q such that Q7'A =’ is the unique Hermite normal form 
(HNF) of A. Since A = MB, we have 


(Q°'M)B = 4’, 


and since Q~'M is non-singular and A’ is in HNF, it follows, by 
the uniqueness of HNF, that 4’ is also the HNF of B.) 
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3.2.4 Row and Column Rank; the Transpose 


This sub-section is just to draw your attention to Proposition 5.2, at the 
bottom of page N55. 


The proof of Proposition 5.2 involves the concept of the transpose of a 
matrix. It is not a difficult concept to understand or remember. For 
example, the transpose of 


12.3 is 1 4 
45 6 2 5). 
3 6 
READ the definition at the bottom of page N55. 
The next exercises develop certain important properties of the transpose. 
It is worth spending ten or fifteen minutes on them if necessary, before 


you look up the answers. If you find the exercises difficult, try some special 
cases (with numbers for the matrix elements) first. 


Exercises 


1. Show that (47)’ = A; i.e. that the transpose of the transpose of 
Ais A. 
2. Exercise 4, page N57. 


Solutions 


1. To find a given entry in the transpose, one simply swaps the 
two suffices around. Doing this twice simply gets back to the 
original entry. Thus (47)? = A. 

This is a case where the argument is almost trivial, but it is 
a little more difficult to find suitable notation with which to 
write out the argument in symbols. If we stick to the conven- 
tion of using lower-case letters for matrix entries, then we can 
denote the (i/)th entry in AT by a;j, and the (ij)th entry in 
(A7)? by aj. Then: 

aj; =a; (for all relevant i, j) 

=a;; (for all relevant i, j) 

Thus (ATY = A. 


2. (a) Let A+ B=C, and denote the (ij)th entries of A, A’, 
B, BT, C, C7 by a;j, aj, etc. Then 


chy = Cy (for all relevant i, j) 
= aji + bj; (for all relevant i, j) 
= ai; + bi, (for all relevant i, j) 


Thus C7 = A7 + BT. 

(b) Let AB = C, and use the same notation convention as 
above. For AB to be defined, A must be (m x n), and 
B must be (7 x p), for some m,n, peZ*. Then C is 
(mm x p). It follows that A’, BT and C7 are (n x m), 
(p x n), (p x m) respectively. We then have 


j= Cy @=1,...p,7=1,...m) 


n 
=} bik My (just putting the previous line 
s in a more recognizable form) 

This shows that C7 = BTAT, 


28 


(c) AA =] 
Therefore 
(A“)"AT = (4A!) (by (b) above) 
sFerl 
Thus (47 ')" is the inverse of AT, i.e. 
(47°) = (AT)! 


“The transpose of the inverse is the inverse of the 
transpose,” 


READ Proposition 5.2 and its proof on pages N55-56. 


Notes 


(i) line 2, page N56 Note the use of Theorem 3.7 from page N48, namely that 
multiplication by a non-singular matrix (here Q~’) does not change the rank of 
a matrix. The result is used again in line 5. 

(ii) line 3, page N56 While we generally distrust statements such as “it is 
obvious” (two weeks’ work), “the student will easily verify that” (a Ph.D dis- 
sertation), ‘it is left as an exercise for the serious student ” (a problem my Ph.D 
supervisor never got around to solving for me), in this particular case it should 
be pretty obvious that the rank of (A’)" is p. A perusal of the matrix labelled 
(5.1) on page N54 shows that all entries below the pth row must be zero, so the 
rank of (4’)T is less than or equal to p. Further, if j < p, the jth row contains, as 
its k,th entry, the only non-zero entry in the kth column, so that it cannot be a 
linear combination of the other rows. Thus, all of the first p rows of A’ are linearly 
independent; i.e., the rank of (A’)? is p. 


Example 
Consider the following matrix in Hermite normal form: 
k ok, ks 
0 @ 0 -2 2 0 0 l 
0 0 ® 1 -4 0 0 2 
A 0 0 0 0 0  @. 2 3 
jo 0 0 0 0 0 0 o0 
0 0 0 0 o 0 0 0 
0 0 0 0 o0 0 0 oO 


There are three columns corresponding to codomain basis elements: 
columns k,, k2, ka shown.: Thus the number of linearly independent 
columns is 3. As for the rows, only the first three are non-zero, so there 
are a maximum of 3 linearly independent rows. But the circled Is in rows 
1, 2 and 3 come in columns that are otherwise composed entirely of zeros. 
Thus none of these rows can possibly be expressed as a linear combination 
of the other rows. So 3 is the exact number of linearly independent rows 
of A’, and is therefore the exact number of linearly independent columns 
of (A')’, which is therefore of rank 3. For convenience, we write out 
(A')? below: 


x a 
Ds 
-00nn o0ðo 


Sook SGoe 
wnQceo0co 
-E-E-E -E-E-E 
ccooocosdéo 
-E-A A-A-A- 


3.2.5 Summary of Section 3.2 


In this section we defined the terms 


standard basis for R” (page C20)* 
Hermite normal form (page N55) 
transpose of a matrix (page N55) 


We introduced the notation 


U, (page N54) 

ky, kascssaka (page N54) 

AT (page N55) 
Theorems 


1. 


(5.1, page N54) 
Given any m x n matrix A of rank p, there exists a non-singular 
m x m matrix Q such that 4’ = Q~'A has the following form: 


(1) There is at least one non-zero element in each of the first p rows 
of A’ and the elements in all remaining rows are zero. 

(2) The first non-zero element appearing in row i(i<p) is a l 
appearing in column k;, where ky < kz < ++: < kp. 

(3) In column k; the only non-zero element is the 1 in row i. 


(Proposition 5.2, page N55) 
The number of linearly independent rows in a matrix is equal to the 
number of linearly independent columns. 


Techniques 


1. 


2. 


Recognize when a matrix is in Hermite normal form, and hence 
determine the rank of such a matrix. 

Given the matrix, A, representing a linear transformation 
oa: U ——> V with respect to specified bases in U and V, determine 
the Hermite normal form of A by selecting a linearly independent 
subset of {o(a,), ..., o(&,)}, {a, ..., @,} being the specified basis in U. 
Use the properties of the transpose of a matrix in matrix manipula- 
tions. 


* C20 means page 20 of this correspondence text. 
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3.3 HOW TO CALCULATE HERMITE NORMAL 
FORM 


3.3.1 Elementary Operations and Elementary Matrices 


This section covers Chapter I, Section 6 of N, which gives a method by 
which Hermite normal form can be calculated in practice. In order to 
prove the viability of the concept of Hermite normal form, we constructed 
the Hermite normal form for a matrix by a method which, though useful 
for demonstrating the existence and uniqueness of Hermite normal form, 
gave no practical method for finding the Hermite normal form for an 
arbitrary matrix. Our method, remember, was to consider the matrix 
to be the matrix of a linear transformation g : U ——> V, with respect 
to a basis {o),...,0,} of U and {B,,..., Bm} of V. We looked through the 
images of the domain basis, and applied the rules: 


(i) If o(a;) is linearly independent of o(«,),..., o(%;-,), take o(«;) as a 
basis element of the codomain. 

(ii) If o(@,) is linearly dependent on o(a,),..., o(¢;-,), then express it in 
terms of the basis elements o(a,,), ..., 0(a%,) which have already been 
chosen. 


However, in the examples we looked at, there was always a rather easy 
“ad hoc” method available of seeing whether o(«;) was a linear com- 
bination of o(a,), ...,0(«;-,), and if so, exactly what the linear combi- 
nation was. This would not be true in general, and we have not yet given 
a general method of answering these questions. It would therefore seem 
at first sight that we have to find such a method, then apply it in order to 
find a Hermite normal form. Actually, as we shall see presently, it is easier 
to go the other way round: Section 6 of Chapter I] in N gives a method of 
obtaining Hermite normal form, which is then immediately applicable to 
finding the linear relations which exist among a set of vectors. 


This method relies on elementary row operations, which you met in Unit 
M100 26, Linear Algebra III, Basically, these operations permute the 
rows of a matrix, or change a given row into a linear combination of 
itself and other rows. Such operations are (as you will shortly see) equiv- 
alent to multiplying the matrix on the left by non-singular matrices, which 
are (see Equation (4.8) on page N51) in turn equivalent to changing the 
codomain basis with respect to which the matrix is expressed (sub-section 
3.1.4). Successive applications of these operations get the matrix closer and 
closer to Hermite normal form, and the fact that each step is a left-multi- 
plication by a non-singular matrix ensures that the codomain basis is 
changed into a new codomain basis each time, with the domain basis kept 
fixed. So we end up with a matrix which is indeed the matrix of the orig- 
inal linear transformation, but expressed with respect to a new codomain 
basis. 


READ Section 6, page N57 up to and including the statement of Theorem 
6.1, page N58. 


Exercises 


1. Try to prove the statement in the sentence immediately preceding 
Theorem 6.1. 

2. Invent an example of a3 x 3 matrix A. Apply three different elementary 
row operations in succession to it. Then apply the same operations to 
the unit 3 x 3 matrix, and check that multiplication of A on the left 
by this matrix has the same result as the application of the operations 
to A. , 

3. If A is an mx n matrix, to which unit matrix would you apply an 
elementary row operation, in order to obtain the requisite elementary 
matrix? (Remember, a unit matrix can be 1x 1,2 x 2,3 x 3, ete.) 
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Solutions 
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1. 


If £ is the elementary matrix representing a given elementary 
row operation, then the effect of that operation on any 
matrix A is, by definition, to transform it into the matrix EA. 
In particular, this is true of the unit matrix /, which is trans- 
formed into the matrix 


El = E. 


Thus E is the matrix obtained by performing the elementary 
Tow operation on Z. 

In exactly the same way, the matrix representing a whole 
sequence of elementary row operations can be'obtained by 
performing this sequence of operations, in the right order, 
on the unit matrix. 

The following is one example: 


12 3 
LetA=]4 5 6}. 
2 4 6 


Consider the effect of applying the following elementary row 
operations, in the following order: 


(i) Interchange rows 1 and 2 
(ii) Multiply row 3 by $ 
(iii) Add (— 1) times row 2 to row 3. 


45 6 
After operation (i), the matrix is |1 2 3]. 
24 6 
45 6 
After operation (ii), the matrix is |1 2 3]. 
12 3 


000 


If we apply the same operations, in the same order, to the 
unit matrix, we obtain 


010 
1 0 O}. 
-10 4 
It is easy to check that 


0 1 O}f! 2 3 456 
1 0 0jj4 5 6ļj=j|i 2 3). 
—1 0 4)[2 4 6 000 


It is mx m, because left-multiplication of A can only be 
defined for a p x m matrix. 


45 6 
After the final operation, the matrix is |1 2 3]. 


LM 3.3.1 


3.3.2 The Use of the Elementary Operations 


READ the rest of Section 6 starting on page N58. (You should find the 
worked examples useful.) 


Note 


Page N6l. AsN Says in the final paragraph on this page, the method described 
is the easiest available hand computation method for finding the inverse of a 
non-singular matrix. 


The following exercises will give you practice in using the algorithm 
described in the reading passage. 


If you wish to use the computer program SMPM2$l, parts (c) and (d) 
of Exercise 1, (c) of Exercise 2, and Exercise 3, are probably worth doing 
this way. Save up all the exercises in this text which you want to do by 
$MPM26! for one trip to the computer terminal. 


Exercises 


1. Obtain the Hermite normal form of the following matrices. 
(a) [2 1 1 (b) [-2 1 -3 
223 1 1 3 
2°10 ol 1 


0 
L4 
@) [2 
1 
3 


L2 


2. Find the inverses of the following matrices, using the method described 


in the last paragraph on page N61. 
() fl 2 3 
23 4 
3 4 6 
3. (Optional) 


(a) [2 1 (b) fo 1 

[3 i 1 Al 
This exercise is designed to show you something about the limitations 
of using a computer for matrix operations. The object is to invert a 
matrix whose rows are nearly, but not quite, linear combinations of 
each other*. A relatively small “‘round-off” error in the initial entries 
leads to a large error in the result. 


PAN WON 
Down Coco 
NOUD o= oo 

om uw 


The matrix in question is f ; It is often called the Hilbert matrix. 
4 34% 
712 —240 180 
Its true inverse is |-28 900 -m Š 
180 —720 600}, 


Evaluate the inverses of 


(a) [0.5 0.333333 0.25 
0.333333 0.25 0.2 
0.25 0.2 0.166667 


0.5 0.333 0.25 
and (b) |0.333 0.25 0.2 |, and compare them. 
0.25 0.2 0.167 


* Such a problem is said to be ill-conditioned (see Unit 8). 
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Solutions 


L @ fl 0 0) ° 
lp to) 
001 

(b) hii 
fan 
000 

©) —1 000 05 
0100 2 
0001 i 
0000 0 

@ fl 230 
0001 
0000 
0000 


2. (a) : (b) fo 1] © [-2 0 1 
1 0 0 3 - 
1 -2 1 

3. (a) You should get something like 


71-9787 —239-92 179-935 
—239-92 899-699 —719-757 
179-935 —719-755 599-802 


quite close to the true inverse. 
(b) You should get something like 


55-49 —177-917 130-005 
—177-917 665-881 —531.12 
130-005 —531:118 447-439 


which is nothing like the true inverse. 


Inverting the Hilbert matrix and other similar problems 
are discussed in Unit 8, Numerical Solution of Simulta- 
neous Algebraic Equations 


3.3.3 Summary of Section 3.3 


„In this section we defined the terms 


elementary row operation (page N57) 
elementary matrix (page N58) 


We introduced the notation 
E,(c), E3,(k), Ey2, ete. (page N57-S8) 
Theorem 


(6.1, page N58) 


Any non-singular matrix A can be written as a product of elementary 
matrices, 


Techniques 


1. Use row operations to reduce a matrix to Hermite normal form. 
2. Use row operations to find the inverse of a matrix. 
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3.4 APPLICATIONS OF HERMITE NORMAL 
FORM 


3.4.1 Linear Problems and Linear Equations 


Systems of linear equations, such as 
4x, + 3x, + 2x3 — x4 = 4 
5x, + 4x. + 3x3 — x4 =4 (1) 
—2x, ~ 2x. — x3 + 2x4 = —3 
lix, + 6x2 + 4x3 + x4 = 11 


arise in many applications of mathematics, of which structural engineering, 
management theory, and electric circuit theory are a few examples. You 
have already seen, in Unit M100 26, Linear Algebra III, how to solve such 
equations by the Gauss elimination method in the case where the matrix 
formed by the coefficients on the left is non-singular; but in this particular 
case the matrix is singular, and the Gauss elimination method breaks 
down. In this section we see how the Hermite normal form can be used to 
solve any linear problem, whether the linear transformation involved is 
singular or non-singular. (It is used to solve the system (1) on pages 
N65-66.) The only restriction is that the vector spaces involved must be 
of finite dimension, so that the linear transformation has a matrix repre- 
sentation. 


The next passage from N reviews the general features of linear problems, 
which you have seen in the Foundation Course, and proceeds to prove a 
theorem which is useful for deciding whether or not a linear problem has a 
non-empty solution set. It then proceeds to show the relevance of Hermite 
normal form to linear problems. Basically, if one reduces the matrix 
associated with the problem to Hermite normal form, then one ends up 
with a new problem which is generally much simpler to solve, and has 
exactly the same solution set as the original problem. 


The theorem proved in this passage is a result which you met in Unit 
M100 26; you should know it and be able to prove it. 


READ Section 7, from page N63, as far as but excluding Theorem 7.2, 
page N66. 


Notes 


(i) Equation (7.1), page N63 {fo} + K(c) means the set of all vectors of the form 
y +8 with y € {Go} (i.e. y = o) and ô € K(a). 

(ii) lines 4, 5, page N64 If B #0, then the solution set cannot be a subspace, 
because it cannot contain the zero vector: o(0) =0 = £. It is a slight stretching 
of the word “dimension ” to talk of the dimension of the solution set in this 
case, but it is obvious what is meant. 

(iii) Jine 6, page N64 Remember that the nullity is the rank of K(c). 

(iv) line 12, page N64 The notation (6;,..., bm) for m x 1 column matrices was 
introduced on page N42. k 
(v) Theorem 7.1, page, N64 The statement “all solutions can be expressed in 
terms of » independent parameters” means that the solution set will be of the form 


{É = fo + xia + ++ + xy a} 


where the “parameters” x1,...,%, take all values in F; i.e. every v-tuple 
(x1, ..., X,) gives an element of the solution set. (In the above expression, éoisa 
particular solution and {c,,..., ay} is a basis of K(o).) 


(vi) line 1, page N65 In reading this paragraph, particularly Equation (7.5), it 
will help if you refer back to the general Hermite normal form (5.1) on page N54. 
One further point which needs a little amplifying is the definition of an 
augmented matrix; it is easy enough to see how the augmented matrix is 
constructed, but just what sort of a linear transformation does it represent? 
The answer is that, though one can of course define a linear transformation 
whose matrix is the augmented matrix, there is no particular point in 
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doing so—the transformation so obtained has no particular significance 
for the problem. This is an illustration of the fact that a matrix is not 
exactly the same thing as a linear transformation. It can be used to repre- 
sent a linear transformation, but it can be put to other uses as well. Other 
cases where a matrix is used which does not directly represent a linear 
transformation, are: 

(a) The “augmenting” of a matrix A to [A, 1] 


aan 1 0-0 
=| asa, 0 17-0 


oe eo 
when using the Hermite normal form method to find the inverse of 
a matrix (see page N61). 


(b) The use of the row-sum check when finding Hermite normal form 
(see also Unit M100 26). This is a way of checking the correctness of 
the arithmetic in row operations; the idea is to include an extra 
column, each entry of which is the sum of all the entries in the corres- 
ponding row. For example, if you want to find the Hermite normal 
form of 


123 4 
3210 
1122 


you should include an extra column consisting of the row sums of the 
original matrix, to get 


1 2 3 4 10 
3 2 1 0 6 
1 1 2 2 6 
since 1 +2 +3 +4 = 10, etc. 


You then find the Hermite normal form for this matrix. Since row 
operations do not disturb existing linear relations among columns, 
when you have finished, the last column should still consist of TOW 
sums. In the above case, if you perform the operations and get 


1 0 oO -1 0 
0 1 0 1 2 
0o o0 1 1 2 


then you can verify that 1 +0 +0-—1 =0,0+140+41 =2, etc., 
So that it is probable that no arithmetical blunders have been made. 


Dropping the check column gives the Hermite normal form of the 
original matrix to be 


1 0 0 -1 
© t © ih 
0 0 1 1 


On the other hand, if at any stage of the calculation the check column 
is not equal to the sum of the others, then you know there is a blunder 
somewhere, and you can rectify it before it does any damage. 


For example, suppose you start with 
1 2 3 4 10 
3 2 1 0 6 
1 1 2 2 6 
as before, and carry out the operations 


row 2—3-rew 1 and row 3— row 1 
to get 
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1 2 3 4 10 
0 -4 -8 -12 -24 


0 1 -Ii -2 - 


Then since 1 — 1 — 2 Æ — 4, there must be a mistake somewhere in 
the last row. 


Exercises 


1. Exercise 2, page N67, using the row sum check in the computations. 
2. Exercise 4, page N68 using either the computer or 

3. Exercise 5, page N68 the row-sum check (or both!) 

4. Exercise 7, page N68. 


Solutions 


1 


The augmented matrix, with row sums as well, is 


row 
sums 
1 2 -3 1 0 1 
f —=1 5 =l 0 6 
2 I 0 1 0 4 
Its Hermite normal form is 
row 
sums 
1 0 1 0 0 | 2 
[e 1-2 0 0 | =i 
0 0 0 1 0 1 


corresponding to the equations 
xX, +x3=0 
Xz — 2x; =0 
X4 = 0. 
The only unknown to appear in more than one equation is x3. 


This can be given an arbitrary value, and the above equations 
then fix x,, x, and x, in terms of x3. 


The solution can be written in the form 


X =X 
X2 = 2x; 
X= X3 
X4 =0 


with x arbitrary (compare the top of page N66). 


The space of solutions is <(—1, 2, 1,0)>. Incidentally, this 
space is the kernel of the corresponding linear transformation. 


The augmented matrix is 
2 =l -3 1 
1 -1 2 -2 
4 -3 1 -3 
1 0-5 3 


whose Hermite normal form is 


1 0 —5 3 
0 1 -7 5 
0 0 0 o0 
0 0 0 0 
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corresponding to the equations 
xı — 5x3 =3 
X2 — 7x3 =5. 


Again x, is the only unknown appearing in more than one 
equation, and so we obtain 


x, =3+5x; 
X,=5+4 7x3 
x3 = x3 


with x, arbitrary. This can also be written 
(X;, X2, X3) = (3, 5, 0) + x3(5, 7, 1) 

and so the solution set (not a subspace this time) is 
(3, 5, 0) + (5, 7, 1). 


The augmented matrix is 


7 3 21 -13 1 -14 
10 3 30 —16 1 -23 
7 2 21 -11 1 -16 
9 3 27 -15 1 -20 


whose Hermite normal form is 


1 0 3 -I 0 -3 
0 1 0 -2 0 2 
0 0 0 0 1 1 
o 0 0 0 0 Ọ 


corresponding to the equations 


xı + 3x3 — x4 = —3 
X — 2x4 =2 
x;=1. 


In this case, x, and x4 can be chosen arbitrarily; we rewrite 
the equations 


X, = —3 — 3x3 +x, 
X2 =2 +2x4 
xs=l 


or 

(Xis X2» X3, X4, X5) = (—3, 2, 0, 0, 1) 

+ (—3, 0, 1, 0, 0)x3 + (1, 2, 0, 1, 0)x4 

giving the solution set 

(—3, 2, 0, 0, 1) + <(—3, 0, 1, 0, 0), (1, 2, 0, 1, 0). 
The associated homogeneous problem is 

= +4y=0. 
Its solution is 

y = C, sin 2x + C; cos 2x. 
The most obvious particular solution is 


y= }sin x. 


3.4.2 Linear Relations Among a Given Set of Vectors 
Example 1 


A brewer uses hops, barley, sugar and water to produce various types of 
beer. He uses them in the following proportions (by weight): 


Beer ce | 


Mild 


Hops JP | Barley / | Sugar € | Water A 


200 
Ordinary Bitter 200 
“Soopa- Broo’ 200 


The prices of the ingredients vary throughout the year, and in calculating 
the cost to him of producing a barrel of each type of beer, he laboriously 
goes through a calculation for each of the three types, once a week. This 
is all quite complicated, as the price of hops is given per ounce, of barley 
per bushel, of sugar per pound and of water per thousand gallons. (He 
looks forward enthusiastically to metrication.) 


Can we help him to save some labour? The answer is yes: the vectors 
(1, 10, 20, 200), (3, 15, 20, 200) and (5, 20, 20, 200), representing the pro- 
portions of ingredients in each type of beer, are linearly dependent. Calling 
them B,, B2, B3 respectively, we in fact have 


Ba = 4B, + B3). 


Since the cost of producing the beer depends linearly on the price of the 
ingredients, it follows that he only has to go through two of his complicated 
calculations, for Mild and “ Soopa-Broo” say. Ordinary Bitter will then 
always be the average of the prices of Mild and “ Soopa-Broo”. 


The general case is not quite so easy. There may be several vectors, and 
one may not know which vectors to try to express as linear combinations 
of which others. Even in the case of three vectors, the situation may be 
more complicated than the one considered above. 


Example 2 
Consider the three vectors in R* 
By = (3, 0, 5, 8), Bs = (1, 2, 3, 4), Bs = (1, —1, 1, 2). 


It is not at all obvious whether $, is a linear combination of $, and $3 
and if so, just what linear combination it is. The secret of the method is 
to write the components of the given vectors as columns of a matrix 


3 1 1 
0 2 -1 
Bels 3 1 
8 4 2 


and to use the fact that elementary row operations on B will not alter 
whatever linear relations exist among these columns*. This fact tells us 
in particular that the row operations that convert B to Hermite normal 
form will not alter these linear relations. 


* Proof Such a linear relation could be written in the form 


3 1 E o A i) 
0 2 —1] Jez} = |0|, ie. BC=0. 
5 3 ills 0 
8 4 2. 
Elementary row operations convert B to B’ = EB where E is non-singular, and hence we 


have B’C = EBC=0 if and only if BC = 0. 
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The Hermite normal form of B is 


1 0 4 
0 I -} 
0 0 0 
0 0 0 


and nothing could be easier than to read off from this that the first two 
columns of B are linearly independent, and that the third is half the first 
minus half the second; in other words, that 


Bs =48, —4f2. 


The same method can be applied to any set of vectors in any finite dimen- 
sional space, provided that they are specified by their coordinates with 
respect to some basis. This general case is discussed further in the next 
reading passage. The first part of the passage (down to “‘an exercise for 
the reader” on page N73) is more complicated than the above treatment, 
because it deals explicitly with the change of basis implied by converting 
B to Hermite normal form, instead of considering only the matrix as we 
have done. Do not spend a lot of time working through this part of the 
argument in N; the most important thing here is not the theory, but how 
to do the calculation. 


READ from “Let B = ...”, page N72 to the end of Section 8, page N74. 


Notes 


(i) Equations (8.2), (8.3), page N72 These give the vectors B,,..., 8, as linear 
combinations of {c,,..., om} (Equation (8.2)) and {c},..., of} (Equation (8.3)). 
By labelling the coordinates with suffices in the manner described, matrices A 
and A’ are constructed from the coordinates in such a way that each column 
gives the coordinates of the corresponding element of the Bs. , 
(ii) line 4, page N73 We have so far been talking in general terms, and have not 
actually specified «3, ..., a4. We now specify that these vectors shall be chosen 
in such a way that the matrix A’ is the Hermite normal form of A. That is to say, 
we select a linearly independent set from the vectors B,,..., Ba. Asin the reading 
passage on page N54, we denote the suffices of these distinguished vectors by 
Kikai... 


Exercises 


I. For each of the following sets of vectors, find a linearly independent 
subset, and express any vectors which are not in this subset in terms 
of those which are. 


(a) (1, 2, 3), (3, 2, 1), (1, 1, 2) 

(b) (1, 2, 3), (3, 2, 1), (1, 1, 1) 

(©) (1, -1, 2), (2, -2, 4), (0, 1, 2), (1, 4, 5) 

Remember to keep a row-sum check in your calculation, if you do it 
by hand. 


2. Express (1, 3, 6, 8) as a linear combination of (2, 3, 4, 5), (1, 2, 3, 4), 
(0, 1, 1, 0) and (0, 0, 1, 1). 


Solutions 


13 1 
1. (a) The Hermite normal form of f 2 I is 
3 1-2 


100 
0 1 Of, and so the three vectors are 
001 


linearly independerit. 


13.1 
(b) The Hermite normal form of k 2 i is 
$ 3 11 


1 0 4 
f 1 i| and so the first two vectors are linearly 
000 


independent, and 


(1, 1, 1) = 4(1, 2, 3) + 4(3, 2, 1). 


1 2 0 1 
(c) The Hermite normal form of |—1 —2 1 4] is 
2 4 2 ~=«5 


1 2 0 1 
0 0 1 3], and so (1, — 1, 2) and (0, 1, 2) are 
o 0 0 0 


linearly independent, and the other two vectors satisfy 
(2, —2, 4) = 2(1, —1, 2) 
(1, 4, 5) =(1, —1, 2) + (0, 1, 2). 


21001 
; 32 1 03 
2. The Hermite normal form of 43116 
5401 8 
1 0 0 0 -1 
. | 0 1 0 0 3 
Slo o 1 0 0 
0 0 0 1 1 
so that 


(1, 3, 6, 8) = —1(2, 3, 4, 5) + 3(1, 2, 3, 4) + (0, 0, 1, 1). 


41 


3.4.3 Summary of Section 3.4 


In this section we defined the terms 


linear problem 

particular solution 

general solution 

associated homogeneous problem 
augmented matrix 

row-sum check 


We introduced the notation 


[4, B] 


Theorem 


(7.1, page N64) 


The system of simultaneous linear equations AX = B has a solution if and 
only if the rank of A is equal to the rank of the augmented matrix [A, B]. 
Whenever a solution exists, all solutions can be expressed in terms of 
v =n — p independent parameters, where p is the rank of A. 


Techniques 


1. Solve AX = B by reducing [A, B] to Hermite normal form. 
2. Determine linear relations (if any) between vectors by reducing the 
matrix of coordinates of those vectors to Hermite normal form. 


3. Use the row-sum check. 
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(page N63) 
(page N63) 
(page N64) 
(page N64) 
(page N64) 
(page C36) 


(page N64) 


+++ 


+ +*+ 


3.5 SUMMARY OF THE UNIT 


Before the complete list of definitions, theorems, techniques and notation, 
here is a review of the unit. 


Given a basis {a,,...,«,} of a vector space U, any vector če U can be 
expressed in terms of {a,,...,0,} by means of a one-column matrix X 
consisting of the components of č. If we change the basis to {a}, ..., «{}, 
we can describe the change by a transition matrix P whose columns are the 
components of the new basis elements in terms of the old. The new one- 
column matrix X” for č is then related to X by the equation 


X = PX’ 


If o: U ——> Visa linear transformation with a matrix A, and we change 
the bases in U and V by transition matrices P and Q respectively, then the 
new matrix for ø is A’ = Q~'AP. The simplest possible form for A‘ 
which we can obtain in this way, has ai, =a), =" = app = 1 (where 
p is the rank of a), and all other matrix entries zero. If we are only allowed 
basis changes in the codomain (i.e. if we can only multiply A on the left 
by non-singular matrices), then the simplest possible form to which we can 
bring A is known as the Hermite normal form; when suitably defined, this 
form is unique. 


The codomain basis {8,,..., 8,,} that achieves Hermite normal form can 
be characterized as follows. Look through the images of the domain 
basis, and apply the following rule. If o(,) is linearly independent of 
o(a;),..., o(@;-4), take o(a,) as the next basis element of the codomain; 
otherwise, reject it. When all the o(a,)s have been exhausted, complete 
the basis of V arbitrarily. 


A matrix A is in Hermite normal form if and only if it obeys the following 
four conditions (compressed into three on pages N54-55). Suppose A is 
mxn. 


(i) There is an integer p, 1 < p < m, such that the first p rows of A are 
not zero rows and the last m — p are. 
(ii) Each of these first p rows has a | as its first non-zero entry. 
(iii) This 1 is in each case the only non-zero entry in its particular column. 
(iv) If this 1 is in the k;th place in the ith row (i=1,...,), then 
ki<k << k. 


If A satisfies all four of the above conditions, then its rank is p. In practice, 
the way to get a matrix A into Hermite normal form is to apply elementary 
row operations repeatedly to it until it is in the required form. It is wisc 
to apply the row operations in an orderly fashion, first of all getting the 
requisite zeros in the first column, then proceeding to the next column, 
and so on. When doing row operations by hand, the row sum check should 
always be used to guard against blunders. 


An elementary row operation can be performed on A by left-multiplying 
A by the matrix which is obtained by applying the same row operation to 
the appropriate unit matrix. By reducing the augmented matrix [A, 7] 
to HNF, one obtains [4’, Q~'], where A’ is the HNF of A and Q` is 
the matrix by which A is left-multiplied to obtain A’. In particular, if A 
is non-singular, one obtains its inverse, A7!, by this means. 


The application of row operations does not change the linear relations that 
exist between the columns of a matrix. Thus, to find linear relations among 
a set of vectors, write these vectors as the columns of a matrix and reduce 
the matrix to Hermite normal form. 


The solution set of a system of simultaneous linear equations is easy to 
find once the set has been converted to an equivalent set of equations by 
reducing the augmented matrix of the system to Hermite normal form. 
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LM 3.5 


As a slight digression from the main theme, we looked at the transpose of 
a matrix, in which the columns become rows and vice versa, and used this 
concept to prove that the row and column ranks of any matrix are equal. 


Definitions 


The terms defined in this unit and page references to their definitions are 
given below. 


matrix of transition (page N50) 
similar (page N52) 
standard basis for R” (page C20) 
Hermite normal form (page N55) 
transpose of a matrix (page N55) 
elementary row operation (page N57) 
elementary matrix (page N58) 
linear problem (page N63) 
particular solution (page N63) 
general solution (page N64) 
associated homogeneous problem (page N64) 
augmented matrix (page N64) 
row-sum check (page C36) 
Theorems 


We list the theorems discussed in this unit. References to the statements of 
the theorems in N are also given. 


1. (4.1, page N52) 
If A is any m x n matrix of rank p, there exist a non-singular n x n 
matrix P and a non-singular m x m matrix Q such that A’ = Q-'AP 
has the first p elements of the main diagonal equal to 1, and all other 
elements equal to zero. 


2. (5.1, page N54) 
Given any mx n matrix A of rank p, there exists a non-singular 
m x m matrix Q such that A’ = Q714 has the following form: 


(1) There is at least one non-zero element in each of the first p rows 
of A’ and the elements in all remaining rows are zero. 

(2) The first non-zero element appearing in row i (i<p)isal 
appearing in column k;, where k, < k, < ++ < Kgs 

(3) In column k; the only non-zero element is the 1 in row i. 


The form A’ is uniquely determined by A. 


3. (Proposition 5.2, page N55) 
The number of linearly independent rows in a matrix is equal to the 
number of linearly independent columns. 


4. (6.1, page N58) 
Any non-singular matrix A can be written as a product of elementary 
matrices. 


5. (7.1, page N64) 
The system of simultaneous linear equations AX = B has a solution 
if and only if the rank of A is equal to the rank of the augmented 
matrix [A, B]. Whenever a solution exists, all solutions can be ex- 


pressed in terms of v =n — p independent parameters, where p is the 
rank of A. 


Techniques 


Determine the matrix of transition for a given change of basis. 

2. Determine the coordinates of a vector with respect to a new basis 
when given the coordinates of that vector with respect to an old basis 
and the appropriate matrix of transition. 

3. Determine the matrix of a linear transformation, ø, from U to V with 
respect to bases A’ in U and B’ in V, given the matrix representing o 
with respect to bases A in U and B in V, and given the form of the 
change of bases A to A’ in U and B to B’ in V. 

4. Recognize when a matrix is in Hermite normal form and hence 
determine the rank of such a matrix. 

5. Given the matrix, A, representing a linear transformation 
a: U ——> V with respect to specified bases in U and V, determine 
the Hermite normal form of A by selecting a linearly independent 
subset of {o(a,),..., o(a,)}, {a,,..., @,} being the basis in U. 

6. Use the properties of the transpose of a matrix in matrix manipula- 
tions. 

7. Use row operations to reduce a matrix to Hermite normal form. 

8. Use row operations to find the inverse of a matrix. 

9. Solve AX = B by reducing [A, B] to Hermite normal form. 

0. Determine linear relations (if any) between vectors by reducing the 

matrix of coordinates of those vectors to Hermite normal form. 

11. Use the row-sum check. 


Notation 
U, (page N54) 
ki, kz, kp (page N54) 
AT (page N55) 
Ez(¢), E3,(k), E12, ete. (page N57-58) 
[A, B] (page N64) 
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3.6 SELF-ASSESSMENT 


Self-assessment Test 


This Self-assessment Test is designed to help you test quickly your under- 
standing of the unit. It can also be used, together with the summary of 
the unit for revision. The answers to these questions will be found on the 
next non-facing page. We suggest you complete the whole test before look- 
ing at the answers. 


1. 
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(i) Let A = {a,, a}, A’ = {ar}, 5} be bases in some vector space U. 
If 
a = 2a, 
and a =a, — 3a, 
what is the matrix of transition from A to A’? 
(ii) Let B= {B,, B2, B3} BY = {81 By, B3} be bases in some other 
vector space V. If 
pi =B: — Ba + 2B, 
B2 = — 2B. + Bs 
Bs = 3B, — Bs, 
what is the matrix of transition from B to B’? 
(iii) Let o: U ——> V, such that 
o(a,) = By — 285 
ala) = —38; + Bo 
What is the matrix representing o with respect to A and B’? 


(iv) Using your answers to parts (i), (ii) and (iii), determine the matrix 
representing o with respect to A’ and B. 


Find the Hermite normal form of 


2 4 1 «10 
1 2 1 =F. 
0 0 1 4 


Use the row-sum check. 
Find a linearly independent subset of 
{(2, 1, 0), (4, 2, 0), (1, 1, 1), (10, 7, 4)} 


and express each of those elements that are not in this subset, as linear 
combinations of those that are. 


1 1 1 
Find the inverse of f 1 — i by determining its Hermite normal 
1 1 2 


form. 


The vectors £,, 82, B3, B4 in R? are written as the columns of a matrix 
B. B,, Bz and B, are linearly independent, and 


Bs = By + Ba — B3 


The second and third rows of the matrix are interchanged, the new 
columns being 


Bi, B2» BS, By 
(i) Are Bi, B3, B3 linearly independent? 
(ii) Which of the following is true? 
(a) By = Bi — B3 + Bs 
(b) B,= Bi +B — BS 
(c) Neither (a) nor (b) holds 


6. Aisa4 x 4 matrix of rank 1. The first column of A contains some non- 
zero elements. Write down the general form of the Hermite normal 
form of A subject to these conditions. 


7. U and V are 3-dimensional vector spaces. With respect to bases 


{a;, %2, a3} of U, {B1, B2, Bs} of V, the matrix of a linear transformation 
a: U — Vis 


14 7 
2 5 81. 
3.6 9 


Write down the matrix of ø with respect to: 
0) {%3 , %2, a} and {81 B2, Bs} 
Gi) {@y, &2, a3} and {$3 , B2; B,}. 
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Solutions to Self-assessment Test 


1. (i) f2 1 (ii) 1 0 - 
[3] |- -2 | 
2 1 -1 
The column entries are the coordinates of the new basis elements in 
terms of the old ones. 
(iii) Since o(a,) = £; + 083 — 283 
(a2) = —3B, + P3 + 083 


the matrix of o with respect to A and B’ is 


1 -3 
0 1 
-2 0 


(iv) The matrix representing o with respect to A’ and B is 
M'=Q7'!MP, where 
P is the matrix of transition from 4 to A’, 
M is the matrix representing o with respect to A and B’, 
Q`" is the inverse of the matrix of transition from B’ to B; 
i.e. Q* is the matrix of transition from B to B’. 


Thus: 
1 -3 
P=[5 al M=| 0 1 
-2 0 
and 
1 0 -3 
Q'=/-1 -2 0 
2 1 -1 
and hence 


2. The Hermite normal form of 


2 4 1 10 
ILa Ii 7 
0 0 4: 


1 
is 

1 2 0 3 

0 0 1 4 

0 0 0 0 
row 
sums 
245 84 
For, row 1 becomes + row 1 gives}1 2 1 7 ill 
001 4 5 
124 5 | s 
row 2 becomes row 2— row 1 gives |0 0 4 2 | 2} 
0014: 5 
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row | becomes row | — row 2, row 3 becomes row 3 — 2 row 2 give 
120316 
004 2 ; 23 
0000} 0 
and finally row 2 becomes 2 row 2 gives 
1203i:6 
0014 ;5 
0000}0 


The vectors in question are just the columns of the matrix in question 2. 
The solution to question 2 gives the required answer. Thus (2, 1, 0) and 
Q, 1, 1) are linearly independent and 


(4, 2, 0) = 2(2, 1, 0) 
(10, 7, 4) = 3(2, 1, 0) + 4(1, 1, 1). 


1 1 1 3 -1 - 
The inverse of |0 1 —-1] is |-1 1 1}. 
1 1 2 -1 0 1 


For, we can find the Hermite normal form of 


1 1 1 1 0 0 
0 1 -1 0 1 0 


1 1 2 0 0 1 


as follows. 
row | becomes row | — row 2 


fl 0 2 1 -l 0 
0 1 =1 0 1 0 
1 1 2 0 0 1 


row 3 becomes row 3 — row 1 


m oo 2 1-1 0 
0 1-1 0 1 0 
fo 1 O-1 1 1 


row 3 becomes row 3 — row 2 


m o 2 1-1 0 
0 1-1 0 1 0 
lo o 1-1 0 1 


Finally, row 1 becomes row | — 2 row 3, and 
row 2 becomes row 2 + row 3 


1 0 0 3 -1 - 
0 l 0 -l1 1 1 
0 0 1 -1 0 1 


The linear relations among f}, B2, B3, B4 are just the same as those 
among ĝi» B2, 83, B4. Thus 

(i) Bis Ba, B3 are linearly independent. 

(ii) By = By + Ba — P3- 

Since the first column of A contains some non-zero elements, the first 
column of the Hermite normal form A’, must be 


ooor 
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But 4’ is of rank 1; hence the other three columns of A’ must be depen- 
dent on this column. So the general form of A’ is 


b 
0 
0 
0 


ooo 
oO00n 


1 
0 
0 
0 


7. (i) Let A = {a,, a, «3} 
A’ = (03, &, oy} 


and B = {B,, Bz, B3} 


Then the matrix of transition from A to A’ is 


001 
010 
100 


and hence the matrix representing ø with respect to A’ and B is 


1 4 Nf0 0 1 741 
2 5 81/0 1 0ļj=]ļ|8 5 2J. 
3 6 941 0 0 9 6 3 


001 
(Note that f 1 1 is the elementary matrix which inter- 
100 


changes columns 1 and 3.) 
(ii) Similarly, if B'={23, 2, B,}, the matrix of transition from B to 


B' is 
001 
010 
100 


and the matrix representing c with respect to A and B’ is 


00 1f 4 7 0 0 17f1 4 7 
010 25 8 0 1 0||2 5 8 
100 3.6 9 1 0 Oj}[3 6 9 

3 6 9 
=|2 5 8 

14 7 
(Again, notice the action of the elementary matrix in interchang- 
ing rows | and 3.) 


Il 
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